Hierarchy-based semantic embeddings for single-valued & multi-valued categorical variables

Abstract In low-resource domains, it is challenging to achieve good performance using existing machine learning methods due a lack of training data and mixed types (numeric categorical). particular, categorical variables with high cardinality pose challenge tasks such as classification regression because requires sufficiently many points for the possible values each variable. Since interpolation not possible, nothing can be learned seen in set. This paper presents method that uses prior knowledge application domain support cases insufficient data. We propose address this by embeddings are based on an explicit representation (KR), namely hierarchy concepts. Our approach 1. define semantic similarity measure between categories, hierarchy—we purely hierarchy-based measure, but other measures from literature used—and 2. use modified one-hot encoding. two embedding schemes single-valued multi-valued perform experiments three different cases. first compare approaches our word pair case. followed creating approaches. A comparison Google, Word2Vec GloVe several benchmarks shows better concept categorisation when knowledge-based embeddings. The third case medical dataset semantic-based standard binary encodings. Significant improvement downstream achieved information.